Classification-relevant Importance Measures for the West German Business Cycle

When analyzing business cycle data, one observes that the relevant predictor variables are often highly correlated. This paper presents a method to obtain measures of importance for the classification of data in which such multicollinearity is present. In systems with highly correlated variables it is interesting to know what changes are inflicted when a certain predictor is changed by one unit and all other predictors according to their correlation to the first instead of a ceteris paribus analysis. The approach described in this paper uses directional derivatives to obtain such importance measures. It is shown how the interesting directions can be estimated and different evaluation strategies for characteristics of classification models are presented. The method is then applied to linear discriminant analysis and multinomial logit for the classification of west German business cycle phases. 1 Important Measures in Classification 1.1 Economic Multipliers Usually, the influence of one variable xi out of a set of variables xj, j = 1, . . . , p, on a function f at a given point x can be measured by the partial derivative ∂f(x)/∂xi. This measure can be interpreted as the change of f , if xi is changed by one unit, given that the other variables xj, j = 1, . . . , p, i 6= j, are held constant (ceteris paribus). This information is only of value, if the variables are uncorrelated. An interesting approach to obtain more reliable measures that can be interpreted like economic multipliers and is based upon averaging over orderings, see Lindeman et al. (1980) and Kruskal (1987) for linear regression and Enache and Weihs (2005) for classification. If multicollinearity is present, the economic multipliers obtained by these methods only reflect the relative importance of the predictors. The effect of changing one predictor variable, e.g. by fiscal policy, on the result is not measured realistically, since the change in one variable inflicts changes in the other variables as well. 1.2 Directional Derivatives It is interesting to analyze the effects of changing one predictor along with the change of all the correlated variables on the result, for example, if one is interested in the effects of a certain fiscal policy action. To incorporate the relationships to all other variables correctly, directional derivatives instead of partial derivatives have to be used. ? This work has been supported by the Collaborative Research Center ‘Reduction of Complexity in Multivariate Data Structures’ (SFB 475) of the German Research Foundation (DFG). 2 ENACHE, WEIHS, and GARCZAREK The directional derivative of function f with respect to x ∈ IR at point x0 in direction d ∈ IR, ||d|| = 1, is defined as: ∂df(x0) ∂dx = lim t→0 f(x0 + td)− f(x0) t , (1) if the limit exists. For practical purposes, the relation between directional derivatives and the gradient can be used: ∂df(x0) ∂dx = d ′ ∂f(x0) ∂x . (2) 1.3 Interpretation of Directional Derivatives Obviously, partial derivatives are a special case of directional derivatives, that is, derivatives in direction of the unit vectors. With directional derivatives, not only the change of one variable xi is analyzed, but the simultaneous change of all variables x1, . . . , xp, described by the direction in d. It can be interpreted geometrically as the slope of the tangential hyperplane at point x0, but unlike partial derivatives, the direction in which the slope is measured is not the coordinate axis of xi but rather the direction described by d. 1.4 Estimation of the Direction Vectors In order to develop importance measures based on directional derivatives, the correct direction vectors have to be chosen. The direction vector corresponding to a change of xj by one unit and the corresponding changes of the other variables according to their correlation with xj is denoted by dj. The single elements of the direction vectors can be estimated using simple linear regressions of all xi on xj: xi = aij + bijxj + εi, i = 1, . . . , p. The coefficients bij can be estimated by sij/sjj and combined to a vector d̃j of estimated slope coefficients which then has to be normalized to unit length, which gives d̂j = d̃j ||d̃j|| =  s1j sjj · sjj √Pp i=1 s 2 ij .. spj sjj · sjj √Pp i=1 s 2 ij  =  s1j √Pp i=1 s 2 ij .. spj √Pp i=1 s 2 ij  = S•j ||S•j|| , (3) that is, the jth normalized column of the empirical covariance matrix. 1.5 Importance Criteria in Classification In classification models with K classes the influence of p predictor variables x on p(k|x), the posterior class probability of class k, is of interest. Since there are K classes, the data has to be split into classes not only to estimate class means and covariance matrices, but also individual class specific direction vectors dki for each variable i. The partial derivatives 1 The prime ′ denotes the matrix transpose and ∂f(x0) ∂x the gradient. Classification-relevant Importance Measures for the West German Business Cycle 3 of selected classification models are given in section 2. The directional derivatives can then easily be computed using equation (2). High positive values meaning a strong increase of the posterior class probability due to the change of the predictor and high negative values indicating a strong decrease. The posterior probability function is usually not linear and therefore the points at which the derivatives are to be evaluated are to be chosen carefully. In classification it is necessary to distinguish different importance criteria, which help choosing the corresponding evaluation strategy. Importance for class characterization This importance measure should give an impression about the influence of the respective variable for staying inside the class, if the class membership is already quite obvious. For this purpose, the direction derivative can be evaluated at the respective class mean. Importance for class separation • evaluate at the borders between classes and average • evaluate at uniformly distributed randomly chosen points which are closest to the class borders and average Such measures should give an impression about which variable has a strong importance for staying inside the class or entering another class. The first strategy would be more precise, but for some classification methods, the class borders cannot be obtained analytically and have to be searched using grid search algorithms, which are computationally demanding for higher dimensions. The second strategy can then be used as an approximation.